{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Prerequisites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Unix Network Programming by late W. Richard Stevens](https://www.douban.com/personage/30082882/)\n",
    "- Know basics of TCP/IP, eg. IP addresses, ports,\n",
    "- Know how to write an echo server with basic sockets APIs.\n",
    "\n",
    "Access to one or more Linux hosts. Sorry for Windows fans.\n",
    "\n",
    "Knows about C++, Python or Go. Sorry for C folks."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# What Unix Network Programming didn't teach"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "W. Richard Stevens planned to write three volumes of UNP2e\n",
    "- Volume 1: Sockets API, published in 1998\n",
    "- Volume 2: IPC, published in 1999\n",
    "- Volume 3: Applications\n",
    "\n",
    "He passed away on Sept 1, 1999, unfortunately\n",
    "\n",
    "Complex protocols, not just formatless echo or short lived daytime\n",
    "\n",
    "Modern concurrent models: *thread-per-connection*, *non-blocking/event-driven/epoll*, *thread pool*, etc.\n",
    "- *POSA2: Patterns for Concurrent and Networked Objects*, pub in 2000\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Highlights of this course"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Focus on server-side TCP network programming\n",
    "- No user interaction, runs for weeks not minutes\n",
    "\n",
    "Measurableperformance\n",
    "- Myths busted\n",
    "\n",
    "No hypothetic optimizations\n",
    "- How many memory (de)allocs kernel does from receiving an Ethernet frame topassing TCP payload to your application?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "常见的误解\n",
    "- 减少动态内存分配\n",
    "- 减少STL的使用"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Layered Network"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ethernet frame\n",
    "\n",
    "IP packet\n",
    "\n",
    "TCP segment\n",
    "\n",
    "Application message"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ethernet frame: 帧\n",
    "\n",
    "IP packet: 组\n",
    "- 一般不去关注IP分片\n",
    "\n",
    "TCP segment: 节\n",
    "\n",
    "Application message: 消息"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Common pitfalls of novices"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Mix business logical with network IO, difficult to unit test\n",
    "\n",
    "Incomplete data received from a TCP peer- Incorrectly determine message boundaries when using TCP\n",
    "\n",
    "Sending C struct to peer\n",
    "- Modify the global align setting causes core dump in 3rd partylibrary\n",
    "\n",
    "TCP self-connection: localhost:54321 <-> localhost:54321\n",
    "\n",
    "Many more in non-blocking network programming"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TCP接受数据不完整\n",
    "- TCP连接断开的时机与条件\n",
    "- 断开过早,可能导致协议栈RST分解, 将连接重置\n",
    "- 阻塞编程: `SO_LINGER`\n",
    "- 非阻塞编程: `SO_LINGER`没用\n",
    "- 在设计TCP协议时要考虑断连\n",
    "\n",
    "\n",
    "\n",
    "直接发送C的struct\n",
    "- 字节对齐, 破坏ABI, 导致第三方API Core Dump\n",
    "- 扩展性差: 增加新字段, 要升级\n",
    "- TCP/IP书的示例\n",
    "  - 定长的字段 \n",
    "  - bitfield\n",
    "\n",
    "\n",
    "\n",
    "TCP自连接\n",
    "- TCP客户端向本机的服务端发送信息, 本机服务端未启动时, 客户端回自己和自己建立连接"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TCP/IP over 1Gb Ethernet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ethernet Frame Structure:\n",
    "- Preamble: 8 Bytes\n",
    "- MAC: 12 Bytes\n",
    "- Type: 2 Bytes\n",
    "- Payload: 46-1500 Bytes\n",
    "- CRC: 4 Bytes\n",
    "- Gap: 12 Bytes\n",
    "\n",
    "\n",
    "Total frame size: 84-1538 Bytes\n",
    "\n",
    "Raw bandwidth: 125MB/s\n",
    "\n",
    "Packet per second:\n",
    "- Maximum packets per second: 1,488,000 (calculated as 125,000,000 / 84)\n",
    "- Minimum packets per second: 81,274 (without jumbo frames)\n",
    "\n",
    "TCP/IP Overhead:\n",
    "- IP header: 20 Bytes\n",
    "- TCP header: 20 Bytes\n",
    "- TCP option (Timestamp): 12 Bytes\n",
    "\n",
    "\n",
    "Maximum TCP throughput: 117,684,752 (calculated as 81,274 × (1500-52))\n",
    "\n",
    "\n",
    "Effective speeds: 112MiB/s or 117MB/s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 关注性能测试的办法"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> ubuntu\n",
    "\n",
    "```shell\n",
    "sudo apt-get update\n",
    "sudo apt-get install netcat pv\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 双机测试"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 本机测试"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```shell\n",
    "$ nc -l 5001 > /dev/null\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "```shell\n",
    "$ dd if=/dev/zero bs=1MB count=1000 | nc localhost 5001\n",
    "1000+0 records in\n",
    "1000+0 records out\n",
    "1000000000 bytes (1.0 GB, 954 MiB) copied, 0.831445 s, 1.2 GB/s\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 本机传输文件测试"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1+0 records in\n",
      "1+0 records out\n",
      "1073741824 bytes (1.1 GB, 1.0 GiB) copied, 8.78293 s, 122 MB/s\n"
     ]
    }
   ],
   "source": [
    "! dd if=/dev/zero of=test.img bs=1G count=1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```shell\n",
    "$ time nc localhost 5001 < test.img\n",
    "\n",
    "real    0m0.005s\n",
    "user    0m0.002s\n",
    "sys     0m0.002s\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "-rwxrwxrwx 1 ningliu ningliu 1073741824 Jun 20 19:27 test.img\n"
     ]
    }
   ],
   "source": [
    "! ls -l test.img"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "vscode": {
     "languageId": "cpp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "214748.36"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "double bandwidth = 1073741824/0.005/1000000;\n",
    "bandwidth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 本机测试, 显示接受速度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "source": [
    "```shell\n",
    "$ dd if=/dev/zero bs=1MB count=10000 | nc localhost 5001\n",
    "10000+0 records in\n",
    "10000+0 records out\n",
    "10000000000 bytes (10 GB, 9.3 GiB) copied, 9.8564 s, 1.0 GB/s\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "source": [
    "```shell\n",
    "$ nc -l 5001 | pv -W > /dev/null\n",
    "6.07GiB 0:00:09 [ 952MiB/s] [               <=>                                                                                ]\n",
    "```\n",
    "\n",
    "```shell\n",
    "$ nc -l 5001 | pv -W > /dev/null\n",
    "9.31GiB 0:00:22 [0.00 B/s] [                   <=>                                                                             ]\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> 可以在测试的时候, 使用`top`检测进程运行情况; `pv`是2进制单位"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 双机测试, 其中一台低配电脑"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Test results with netcat\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "118MB/s between atom and {e6400, e350} : Way too slow!\n",
    "\n",
    "客户端: \n",
    "- (内核态)`/dev/zero`->(用户态)`dd`\n",
    "- (用户态)`dd`->(内核态)`pipe`\n",
    "- (内核态)`pipe`->(用户态)`nc`\n",
    "- (用户态)`nc`->(内核)TCP\n",
    "\n",
    "服务端\n",
    "- (内核态)TCP->(用户态)`nc -l`\n",
    "- (用户态)`nc -l`->(内核态)`/dev/null`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "580MB/s on atom, dd | nc, nc -l > /dev/null\n",
    "\n",
    "客户端: \n",
    "- (内核态)`pipe`->(用户态)`nc`\n",
    "- (用户态)`nc`->(内核)TCP\n",
    "\n",
    "服务端\n",
    "- (内核态)TCP->(用户态)`nc -l`\n",
    "- (用户态)`nc -l`->(内核态)`/dev/null`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1074MB/s on atom, nc< file, nc -l > /dev/null "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Basic, non-concurrent examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- TTCP: classic TCP performance testing tool\n",
    "- Round-trip: measure clock error between two hosts\n",
    "- Netcat: a Swiss knife\n",
    "- Slow sink/source"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- TTCP: TCP性能测试工具, 协议与`nc`不同\n",
    "- 测试两台机器之间的时间差, 使用UDP\n",
    "- 网络编程的瑞士军刀, 与标准输入输出打交道\n",
    "- 慢速收发的工具, 应用层模拟网速比较慢的情况, 特别字非阻塞情况下"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Concurrent examples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- SOCKS proxy server\n",
    "\t- Relay two TCP connections\n",
    "- Sudoku solver\n",
    "\t- A lot services fit in thisrequest-response model\n",
    "- Simple memcached\n",
    "- Broadcasting to multiple TCP peers\n",
    "\t- How to deal with slow receiver?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 阻塞情况较容易; 非阻塞情况下, 带宽不一致\n",
    "- 请求响应式模型; 连接之间是独立的; 服务器并发支持; 利用CPU多核资源\n",
    "- 非阻塞网络编程例子\n",
    "- 应用层TCP广播; 连接之间有交互的, 一个消息要发给多个TCP的接收端; 连接生命期管理; 如果一个连接收的速率慢会发生什么?数据会堆积, 还是会造成服务器内存使用暴涨."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data processing with multiple machines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- Parallel N-queues\n",
    "- Median of numbers across machines\n",
    "- Frequent queries\n",
    "- Distributed sorting"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "之前的例子,算是网络编程例子, 没有应用价值; 现在这个例子使用多台机器的数据处理, 例如hadoop, Spark等框架"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 并行的8皇后问题\n",
    "- 求分布在多台机器上数的中位数\n",
    "- 查询出现测试最多的项\n",
    "- 分布式排序; MapReduce中的Shuffle\n",
    "\n",
    "数据交换越来越多"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Advanced topics"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- RPC - A basic building block for various servers\n",
    "- Load balancing\n",
    "  - Better than round-robin\n",
    "- Capacity ofa serving system\n",
    "  - How many machines do I need to support X QPS?\n",
    "  - What will be the number of replicas of each component?\n",
    "- Fight for (tail) latency\n",
    "  - Mean and Percentiles: 95%, 99%"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- RRC: 不与IO打交道\n",
    "- 负载均衡\n",
    "- 服务器的容量管理\n",
    "- 延迟: 如何测试; 测试标准"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "C++17",
   "language": "C++17",
   "name": "xcpp17"
  },
  "language_info": {
   "codemirror_mode": "text/x-c++src",
   "file_extension": ".cpp",
   "mimetype": "text/x-c++src",
   "name": "C++17",
   "version": "17"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
